Apparatus, method and program for image conversion

ABSTRACT

Disclosed is a conversion apparatus including a decoding unit that decodes input encoded data into image data, and a size conversion unit that converts size of the image data based on a size conversion request received from outside and that expands/contracts, or splits/synthesizes motion vector or prediction mode of encoding parameters, received from the decoding unit, based upon a size conversion request, and an encoding unit that encodes image data after size conversion using the prediction mode or motion vector converted and transmits the resulting encoded image data via a transmission buffer to outside.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is the National Phase of PCT/JP2010/050083,filed Jan. 7, 2010, which claims priority rights based upon JP PatentApplication JP 2009-002989 filed on Jan. 8, 2009. The total contents ofthe JP Patent Application of the senior filing date are to beincorporated and disclosed by reference in the present specification.

TECHNICAL FIELD

This invention relates to a technique of picture conversion and, moreparticularly, to a method, an apparatus and a program that may beapplied to advantage to re-encoding to be executed at the time ofconversion in size of a bit stream obtained on encoding a moving picturefor compression.

BACKGROUND

In these days, H.261 and H.263, recommended by ITU-T (InternationalTelecommunication Union Telecommunication Standardization Sector), andMPEG-4 (Moving Picture Experts Group Phase 4), internationallystandardized by ISO/IEC (International Organization forStandardization/International Electrotechnical Commission), have beenknown as moving picture compression encoding systems for efficientlytransmitting a moving picture signal at a low bit rate. Also, H.264/MPEG-4 AVC (Advanced Video Coding), internationally standardized by theITU-T and ISO/IEC, is attracting notice as being a system capable oftransmitting a moving picture signal more efficiently than the abovementioned moving picture compression encoding systems.

The moving image compression encoding system uses intra-prediction thatexecutes encoding using only image data of the current frame andinter-prediction that executes encoding using images of a past frameand/or a future frame.

With the intra-prediction, DCT (Discrete Cosine Transform) is applied inencoding to an input image frame, in terms of a macro-block as a unit,to generate DCT coefficients, which DCT coefficients are then encoded byvariable length encoding. Or, prediction is carried out on a permacro-block basis or on a per block basis obtained on furthersubdividing the macro-block, using pixels of near-by blocks, such asleft or upper blocks: DCT or DIT (Discrete Integer Transform) is thenapplied to a prediction residual signal to generate DCT or DITcoefficients, which are then encoded by variable length encoding.

With the inter-prediction, in encoding an input image frame, motioncompensation prediction is executed with decoded pixels of past and/orfuture frames to find a prediction residual signal. DCT or DIT is thenapplied to the residual signal, after which the motion vector as well asDCT coefficients are encoded by variable length encoding.

In these days, such a system that interconnects terminals of differentsorts, such as mobile phones or PCs, or terminals of the same sort butdifferent in functions, is being discussed.

In such interconnection, there is a difference in capabilities ofterminals, depending upon the sorts of the terminals, such as mobilephones or PCs, or upon functions of the terminals. Thus, datacommunication which is adapted to different terminals is required.

In data communication of moving images, image size that may be displayedmay differ from one terminal to another because of difference inlimitations imposed by respective terminals. It is thus necessary toprovide a conversion apparatus that converts an encoded bit stream of amoving image (encoded data) to one which fits in with a terminal ofinterest.

However, to execute size conversion of received encoded data, it isnecessary to decode the received encoded data, to convert the image sizeand finally to re-encode the moving image signal having image sizechanged.

Moreover, in case a network of destination of connection differs inbandwidth, in particular, is narrower in bandwidth, it is necessary tosuppress the bit rate of the encoded data. However, if simply the bitrate is suppressed, the image quality is degraded. It is thus necessaryto exercise caution to maintain the code amount per block in each framesuch as by contracting the image size or by decreasing the frame rate.

In Patent Document 1 (JP Patent Kokai Publication No. JP-P2002-142222A),there is disclosed an image size changing device in which an imagecompression-encoded along a spatial axis and along a time axis mayspeedily be converted into image encoded with different resolution. Avariable length decoding means decodes a compressed image to output anorthogonally-transformed image and motion vector information for eachframe. The orthogonally-transformed image is then subjected to inversequantization. A layered inverse orthogonal transform means executesinverse orthogonal transform to the orthogonally-transformed image inaccordance with a change rate of the number of pixels to output anon-compressed difference image or a non-compressed changed inresolution. A motion vector correction means corrects the decoded motionvector information in accordance with a change rate of the number ofpixels. A motion vector generation means generates motion vectorinformation after the resolution change using the decoded motion vectorinformation. A motion compensation non-compressed image generation meansthen uses the motion vector information after the resolution change toexecute the processing of motion compensation of the non-compresseddifference image to generate a non-compressed image. A motioncompensation non-compressed difference image generation means uses themotion vector information after the resolution change to generate anon-compressed difference image or the non-compressed image. Thenon-compressed difference image or the non-compressed image is thencompressed on encoding. The non-compressed image following theresolution change may directly be generated from the compressed imageprevious to the resolution change.

In Patent Document 2, there is disclosed an encoded data image sizeconversion apparatus whereby the image size of data encoded usingprediction of motion compensation and orthogonal transform may readilybe changed with improved image quality. With the encoded data image sizeconversion apparatus, macro-block image data, converted from inputencoded image data to a preset image size, may be obtained by an imagedata conversion unit. A motion vector of the encoded image data suppliedis multiplied by a motion vector conversion unit with a presetconversion ratio to obtain a converted motion vector for the macro-blockimage data. In case a conversion ratio is less than 1, an integratedmotion vector is found depending upon the variance of a plurality ofconverted motion vectors necessary for conversion of the image data.Encoded data of a size-changed image is obtained using the integratedmotion vector and the macro-block image data as found.

In Patent Document 3, there is disclosed a digital moving image decodingapparatus wherein an output image size may be changed by a simplifiedconfiguration. The digital moving image decoding apparatus is notsusceptible to image quality degradation even in an image portion whichmoves intensely. Moreover, it is unnecessary to perform the processingof low pass filtering or decimation following the processing ofdecoding. The decoding apparatus includes a motion vector conversionunit that converts the value of the motion vector from the motion vectorinformation included in a bit stream. In case the magnitude of themotion vector is small, inverse DCT is performed using only DCTcoefficients of a lower frequency side and, in case the magnitude of themotion vector is large, inverse DCT is performed using not only DCTcoefficients of the lower frequency side but also DCT coefficients of ahigh frequency side.

In Patent Document 4, there is disclosed an encoding apparatus thatexecutes re-encoding in accordance with an encoding scheme which is acombination of motion compensation prediction and DCT. With the encodingapparatus, the calculations amount to be performed when encoded secondvideo data with the changed image size is to be obtained from encodedfirst video data may be reduced on the whole. In decoding the encodedinput data, necessary for re-encoding, the image size is converted inthe DCT domain in connection with prediction error. It is then necessaryto perform IDCT (Inverse DCT) processing followed by decoding into imagedata by motion compensation prediction in the spatial domain. The volumeof calculations in IDCT processing and image size conversion may thus bedecreased to prevent the volume of calculations in motion compensationprediction decoding in the DCT domain from increasing.

-   -   Patent Document 1: JP Patent Kokai Publication No.        JP-P2002-142222A    -   Patent Document 2: JP Patent Kokai Publication No.        JP-P2002-344973A    -   Patent Document 3: JP Patent Kokai Publication No.        JP-P2001-112002A    -   Patent Document 4: JP Patent Kokai Publication JP-P2002-374536A

SUMMARY

The following is an analysis by the present invention.

In converting image size or frame rate, a most problem is a calculationamount for re-encoding. It is because calculation for re-prediction isnecessary in encoding image data after image size conversion of areceived stream.

On the other hand, in re-encoding after the image size conversion of areceived stream, the usage of information of the received streamdeteriorates the image quality.

It is an object of the present invention to provide an apparatus, amethod and a program for speeding up conversion of image size of inputencoded data.

It is another object of the present invention to provide an apparatus, amethod and a program for suppressing image quality from beingdeteriorated in converting image size of input encoded data.

It is a further object of the present invention to provide an apparatus,a method and a program for speedily converting frame rate of inputencoded data. It is yet another object of the present invention toprovide an apparatus, a method and a program for suppressingdeterioration in image quality when converting the frame rate of theinput encoded data.

To accomplish at least one of the above mentioned objects, the inventiondisclosed in the present Application substantially has the followingconfiguration, though not limited thereto.

In one aspect of the present invention, there is provided an imageconversion apparatus including a decoding unit that decodes encoded datareceived, a size conversion unit that change the size of image datadecoded, and an encoding unit that in encoding the image data after sizechange re-uses, for the encoding, at least one of prediction mode, blocktype, motion vector and the reference frame information of the encodeddata received and a conversion control unit that receives sizeconversion request from outside said apparatus and encoding parametersupplied from said decoding unit to control said size conversion unitand said encoding unit.

In another aspect of the present invention, there is provided an imageconversion method comprising:

-   -   decoding encoded data received,    -   changing the size of image data decoded,    -   re-using, in encoding the image data after size conversion, at        least one of prediction mode, block type, motion vector and        reference frame information of the encoded data received, for        the encoding, and    -   controlling the size conversion and the encoding, based on size        conversion request received from outside and encoding parameter        from the decoding.

In yet another aspect, the present invention provides a program or acomputer readable recording media storing therein a program causing acomputer to execute the processing comprising:

-   -   decoding encoded data received, the processing of changing the        size of image data decoded,    -   re-using, in encoding the image data after size conversion, at        least one of prediction mode, block type, motion vector and        reference frame information of the encoded data received, for        the encoding, and    -   controlling the size conversion and the encoding, based on size        conversion request received from outside and encoding parameter        from the decoding.

According to the present invention, image size of input encoded data maybe converted speedily. Moreover, according to the present invention,image quality may be prevented from being deteriorated in the conversionof the image size of the input encoded data.

Still other features and advantages of the present invention will becomereadily apparent to those skilled in this art from the followingdetailed description in conjunction with the accompanying drawingswherein only exemplary embodiments of the invention are shown anddescribed, simply by way of illustration of the best mode contemplatedof carrying out this invention. As will be realized, the invention iscapable of other and different embodiments, and its several details arecapable of modifications in various obvious respects, all withoutdeparting from the invention. Accordingly, the drawing and descriptionare to be regarded as illustrative in nature, and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a configuration of a moving image conversionapparatus of Exemplary Embodiment 1 of the present invention.

FIG. 2 is a diagram showing a configuration of an encoding unit ofExemplary Embodiments 1 to 4 of the present invention.

FIG. 3 is a diagram showing a configuration of a moving image conversionapparatus of Exemplary Embodiment 2 of the present invention.

FIG. 4 is a diagram showing a configuration of a moving image conversionapparatus of Exemplary Embodiment 3 of the present invention.

FIG. 5 is a diagram showing a configuration of a moving image conversionapparatus of Exemplary Embodiment 4 of the present invention.

FIG. 6 is a diagram showing a configuration of a moving image conversionapparatus of Exemplary Embodiment 5 of the present invention.

FIG. 7 is a diagram showing a configuration of an encoding unit ofExemplary Embodiment 5 of the present invention.

FIG. 8 is a diagram showing a configuration of a moving image conversionapparatus of Exemplary Embodiment 6 of the present invention.

FIG. 9 is a diagram showing a configuration of an encoding unit ofExemplary Embodiment 6 of the present invention.

FIG. 10 is a diagram showing a configuration of a moving imageconversion apparatus of Exemplary Embodiment 7 of the present invention.

FIG. 11 is a diagram showing a configuration of an encoding unit ofExemplary Embodiment 7 of the present invention.

FIG. 12 is a diagram showing a configuration of a moving imageconversion apparatus of Exemplary Embodiment 8 of the present invention.

FIG. 13 is a diagram showing a configuration of an encoding unit ofExemplary Embodiment 8 of the present invention.

FIG. 14 is a diagram showing a configuration of a moving imageconversion apparatus of Exemplary Embodiment 9 of the present invention.

FIG. 15 is a diagram showing a configuration of an encoding unit ofExemplary Embodiment 9 of the present invention.

FIG. 16 is a diagram showing a configuration of a moving imageconversion apparatus of Exemplary Embodiment 10 of the presentinvention.

FIG. 17 is a diagram showing a configuration of an encoding unit ofExemplary Embodiment 10 of the present invention.

FIG. 18 is a diagram showing a configuration of a moving imageconversion apparatus of Exemplary Embodiment 11 of the presentinvention.

FIG. 19 is a diagram showing a configuration of a moving imageconversion apparatus of Exemplary Embodiment 12 of the presentinvention.

PREFERRED MODES

According to the present invention, there are provided a means thatdecodes encoded data received, a means that changes the size of imagedata decoded, and a means that re-uses, in encoding the image data aftersize change, at least one of a prediction mode, a block type and amotion vector of the encoded data received, for the encoding.

More specifically, a moving image conversion apparatus (100 of FIG. 1)according to the present invention includes a reception buffer (102 ofFIG. 1) that receives input encoded data (110 of FIG. 1), a decodingunit (103 of FIG. 1) that decodes the encoded data received from thereception buffer (102) into image data, a conversion control unit (101of FIG. 1) that receives a size conversion request (107 of FIG. 1) fromoutside the apparatus and a set of encoding parameters (112 of FIG. 1)from the decoding unit (103 of FIG. 1) to control a size conversion unit(104 of FIG. 1) or an encoding unit (105 of FIG. 1), a size conversionunit (104) that converts the size of image data entered from thedecoding unit (103) under a command from the conversion control unit(101), an encoding unit (105) that generates the encoded data from theimage data supplied from the size conversion unit (104) under a commandfrom the encoding unit (101), and a transmission buffer (106 of FIG. 1)that receives the encoded data from the encoding unit (105) to outputthe received encoded data.

The moving image conversion apparatus (100 of FIG. 1) according to thepresent invention, decodes the input encoded data in the decoding unit(103) into image data. Based upon the size conversion request receivedfrom outside, the size conversion unit (104) converts the size of theimage. The moving image conversion apparatus then performsexpansion/contraction and splitting/synthesis of motion vector orprediction mode, which is an encoding parameter obtained from thedecoding unit (103), based upon the conversion ratio of the sizeconversion request. The encoding unit (105) encodes the image data aftersize conversion, using converted prediction mode or motion vector, andsends the encoded image data via the transmission buffer (106) tooutside the moving image conversion apparatus. In the present invention,it is possible to provide a conversion system for speedily convertingthe image size of the input encoded data. Moreover, it is possible toprovide a conversion system in which the image size of the input encodeddata may be converted, using the input encoded data, in a mannersubstantially suppressing deterioration of image quality. The presentinvention will now be described with reference to Exemplary Embodiments.

Exemplary Embodiment 1

FIG. 1 shows a configuration of a moving image conversion apparatus 100according to Exemplary Embodiment 1 of the present invention. Referringto FIG. 1, a moving image conversion apparatus 100 includes a conversioncontrol unit 101, a reception buffer 102, a decoding unit 103, a sizeconversion unit 104, and encoding unit 105 and a transmission buffer106.

Based upon a size conversion request 107 received from outside theapparatus and encoding parameters 112 from the decoding unit 103, theconversion control unit 101 sends a size conversion request 108 to thesize conversion unit 104, and sends conversion information 109 to theencoding unit 105. The size conversion request 107 may include, forexample,

-   -   multiplication factor;    -   image size following size conversion;    -   bit rate; and    -   encoding information.

The conversion control unit 101, on receipt of encoding information fromoutside as the size conversion request 107, is able to send it to thedecoding unit 103.

The following describes specifically the size conversion request 108which is a command transmitted from the conversion control unit 101 tothe size conversion unit 104.

On receipt of the size conversion request 107 from outside, theconversion control unit 101 sends the size conversion request 108 forconverting the input image data size to the size conversion unit 104.

In case the size conversion request 107 from outside includes amultiplication factor, the size conversion request is transmittedunchanged to the size conversion unit 104.

In case the image size after conversion is supplied from outside, theconversion control unit 101 gets an image size before size conversionfrom the encoding parameters 112 received from the decoding unit 103 andsends the information on the multiplication factor or on theinput/output image size to the size conversion unit 104. In case theimage size after size conversion is the same as the image size beforeconversion, the conversion control unit 101 commands the size conversionunit 104 not to convert the image size.

The following describes specifically the conversion information 109which is a command from the conversion control unit 101 to the encodingunit 105.

Using the size conversion request 107 from outside, and the set ofencoding parameters 112, supplied from the decoding unit 103, theconversion control unit 101 generates the conversion information 109 tobe sent to the encoding unit 105.

The conversion information 109 may include, for example,

-   -   image data size;    -   frame type;    -   prediction mode and direction of prediction (direction of        prediction)    -   motion vector;    -   reference frame information;    -   block size;    -   block type; and    -   bit rate.

For inter-frame prediction, the following information in the conversioninformation 109 are used.

-   -   frame type;    -   motion vector;    -   reference frame information;    -   block size; and    -   block type.

For intra-frame prediction, the following information in the conversioninformation 109, are used.

-   -   frame type;    -   prediction mode and direction of prediction;    -   block size; and    -   block type.

The following describes the sequence of operations for generating theconversion information 109 using the size conversion request 107 andencoding parameters 112.

For simplicity of explanation, the cases of the multiplication factor of2 and ½ are taken by way of examples. Of course, these cases are for thesake of explanation only and are not for limiting the invention. As willbe apparent from the principle of the present invention, using othermultiplication factors, motion vector, prediction mode/direction orblock size, may similarly be possible.

<Case 1> the Case of Doubling the Image Size (×2)

In case a motion vector of an 8×8 block A of input encoded data is fiverightwards and four upwards, the information that the block is sized16×16 and a motion vector is five rightwards and four upwards is sent tothe encoding unit 105, in order to encode the block A after sizeconversion. The reference frame information has the same value as thatof the input encoded data.

<Case 2> the Case of Halving the Image Size (×½)

In case the intra-frame prediction mode in an 8×8 block B of inputencoded data is for leftward direction, the information that the blockis sized 4×4 and that the intra-frame prediction mode is leftwarddirection is sent to the encoding unit 105, in order to encode the blockB after size conversion.

The reception buffer 102 receives input encoded data 110, sent theretofrom outside, and sends the received data to the decoding unit 103.

In case the encoding information 117 is notified from the conversioncontrol unit 101, the decoding unit 103 decodes input image data 111received from the reception buffer 102 to generate decoded image data113 using the encoding information 117. The so decoded image data issent to the size conversion unit 104.

In case the encoding information is not notified from the conversioncontrol unit 101, the decoding unit 103 decodes the input encoded data,using the encoding information included in the input encoded data 111,to generate decoded image data.

The decoding unit 103 also notifies the conversion control unit 101 ofencoding parameters 112 of the input encoded data obtained on decoding.

Using the multiplication factor or the input/output image size, includedin the information of the size conversion request 108 from theconversion control unit 101, the size conversion unit 104 converts theimage size of the image data 113 received from the decoding unit 103 andsends the so converted image data to the encoding unit 105.

Using the information from the conversion control unit 101, the encodingunit 105 encodes the image data received from the size conversion unit104, and sends the so encoded data 115 to the transmission buffer 106.

In the present Exemplary Embodiment, the processing by the conversioncontrol unit 101, the decoding unit 103, the size conversion unit 104and the encoding unit 105 may be implemented by a program which isexecuted on a computer constituting the moving image conversionapparatus 100.

FIG. 2 shows the configuration of the encoding unit 105 of FIG. 1.Referring to FIG. 2, the encoding unit 105 will be described. Althoughthe case of using H.264, is taken up for explanation in the presentExemplary Embodiment, it should be noted that this is merely an examplepresented for illustration of the present invention and not for limitingthe invention. As will be apparent from the principle of the movingimage conversion system, the present invention may operate with anyother suitable moving image compression encoding systems.

Referring to FIG. 2, the encoding unit 105 includes an inter-frameprediction data generation unit 201, an intra-frame prediction datageneration unit 202, a switch 203, a frame memory 204, a DCT unit 205, aquantizer 206, an inverse quantizer 207, an inverse DCT unit 208 and anin-loop filter 209. The encoding unit 105 also includes a variablelength encoding unit 210 and a header information generator 211.

Using conversion information 109, received from the conversion controlunit 101, and image data 213 from the frame memory 204, the inter-frameprediction data generation unit 201 performs inter-frame prediction, andgenerates prediction data which is sent to the switch 203. Theconversion information 109 used in the inter-frame prediction datageneration unit 201 may include:

-   -   motion vector;    -   reference frame information;    -   block size; and    -   block type.

Using the conversion information 109, received from the conversioncontrol unit 101, and the image data 114, received from the sizeconversion unit 104, the intra-frame prediction data generation unit 202performs intra-frame prediction to generate prediction data which issent to the switch 203. The conversion information 109, used in theintra-frame prediction data generation unit 202, may include:

-   -   prediction mode and direction of prediction;    -   block size; and    -   block type.

The switch 203 changes over prediction data, generated by theinter-frame prediction data generation unit 201 or the intra-frameprediction data generation unit 202, in accordance with the conversioninformation 109 as instructed by the conversion control unit 101 in sucha manner that the prediction data will be able to be sent to the DCTunit 205, in-loop filter 209 and the intra-frame prediction datageneration unit 202.

The quantizer 206 determines a quantization step, using the bit rateincluded in the conversion information 109 from the conversion controlunit 101 as a goal. The method of deciding on the quantization step issimilar to that for a normal encoding unit and hence is not explained.

Based upon the conversion information 109, received from the conversioncontrol unit 101, the header information generator 211 generates theheader information 212, which is output to the variable length encodingunit 210.

The conversion information 109 used by the header information generator211 may include:

-   -   frame type;    -   image size;    -   motion vector;    -   block size;    -   block type;    -   prediction mode and direction of prediction; and    -   reference frame information.

Except for those described above, the configuration and the operation ofFIG. 2 are similar to those of the routine encoding unit and hence thecorresponding explanation is dispensed with.

Referring to FIG. 1, the transmission buffer 106 outputs an image streamreceived from the encoding unit 105 to outside.

In the present Exemplary Embodiment, the image size of input encodeddata may be speedily converted.

Exemplary Embodiment 2

The following describes Exemplary Embodiment 2 of the present invention.FIG. 3 shows a detailed configuration of a moving image conversionapparatus 300 of Exemplary Embodiment 2 of the present invention. InFIG. 3, the elements which are the same as those of FIG. 1 are depictedby the same reference numerals. In the following, the description of theelements which are the same as those of Exemplary Embodiment 1 is notmade to avoid redundancy. Also, an encoding unit 105 is the same inconfiguration as that of FIG. 1.

A conversion control unit 301 sends a size conversion request 108 to asize conversion unit 104, and sends conversion information 109 to anencoding unit 105, based upon a size conversion request 107 from outsideand upon encoding parameters 112 from a decoding unit 103.

The sequence of operations of preparing the conversion information 109using the size conversion request 107 and the encoding parameters 112will now be described. Although the multiplication factor is set to twofor simplicity, by way of an example, any suitable multiplication factorother than two may be used.

<Case 1> Case of Doubling the Image Size (×2)

In case a motion vector of an 8×8 block A of input encoded image data isfive rightwards and four upwards, and the block A after size conversionis to be encoded, a block sized 16×16 and a motion vector 10 rightwardsand 8 upwards are sent to the encoding unit 105. The same values areused for the reference frame information as those of the input encodeddata.

Except for those described above, the configuration and the operationare the same as those of Exemplary Embodiment 1, and hence thecorresponding explanation is dispensed with. In the present ExemplaryEmbodiment, the image size of the input encoded data may be convertedspeedily. As compared to the above mentioned Exemplary Embodiment 1,theencoding efficiency may be improved in the present Exemplary Embodiment.The processing of the conversion control unit 301, decoding unit 103,size conversion unit 104 and the encoding unit 105 may be implemented inthe present Exemplary Embodiment on a program which is executed on acomputer of the moving image conversion apparatus 300.

Exemplary Embodiment 3

The following describes Exemplary Embodiment 3 of the present invention.FIG. 4 shows a detailed configuration of a moving image conversionapparatus 400 of Exemplary Embodiment 3 of the present invention. InFIG. 4, the elements which are the same as those of FIG. 1 are depictedby the same reference numerals. In the following, the description of theelements which are the same as those of Exemplary Embodiment 1 is notmade to avoid redundancy. Also, an encoding unit 105 is the same inconfiguration as that of FIG. 2.

A conversion control unit 401 sends a size conversion request 108 to asize conversion unit 104, and sends conversion information 109 to theencoding unit 105, based upon a size conversion request 107 from outsideand encoding parameters 112 from a decoding unit 103.

The sequence of operations of preparing the conversion information 109using the size conversion request 107 and the encoding parameters 112will now be described. Although the multiplication factor is set to twofor simplicity, by way of an example, it is possible to use anymultiplication factor other than four.

<Case 1> Case of Quadrupling the Image Size (×4)

In case the intra-frame prediction mode of a 4×4 block B of inputencoded image data is in the left upward direction, with an angle being30°, the block size is set to 16×16 in order to encode thesize-converted block B. If the left upward direction, with the angle of30°, is not provided in the intra-frame prediction mode of the 16×16block, approximation is made to an existing prediction direction. Forexample, if there are only four directions of leftward, rightward,upward and downward, the information for the leftward direction is sentto the encoding unit 105.

The configuration as well as the operation other than those describedabove is the same as those of the above described Exemplary Embodiment1, and hence the corresponding explanation is dispensed with. In thepresent Exemplary Embodiment, the operation and the meritorious resultmay be displayed that the image size of the input encoded data mayexpeditiously be converted, while size conversion may be improved inflexibility. In the present Exemplary Embodiment, the processing by theconversion control unit 401, decoding unit 103, size conversion unit 104and the encoding unit 105 may be implemented by a program which isexecuted on a computer of the moving image conversion apparatus 400.

Exemplary Embodiment 4

The following describes Exemplary Embodiment 4 of the present invention.FIG. 5 shows a detailed configuration of a moving image conversionapparatus 500 of Exemplary Embodiment 4 of the present invention. InFIG. 5, the elements which are the same as those of FIG. 1 are depictedby the same reference numerals. In the following, the description of thesame elements as those of Exemplary Embodiment 1 is dispensed with inorder to avoid redundancy. Also, an encoding unit 105 is the same inconfiguration as that of FIG. 2.

A conversion control unit 501 sends a size conversion request 108 to asize conversion unit 104, and sends conversion information 109 to theencoding unit 105, based upon a size conversion request 107 from outsideand encoding parameters 112 from a decoding unit 103.

The sequence of operations of preparing the conversion information 109using a size conversion request 107 and the encoding parameters 112 willnow be described. It is noted that the multiplication factor is set totwo (×2) and one-half (×½) only by way of illustration. As will beapparent from the principle of the present invention, the cases of usingother values of the multiplication factor or changing the motion vector,prediction mode/direction of prediction or the block size may similarlybe dealt with by the present invention.

<Case 1> the Case of Doubling the Image Size (×2)

In case a motion vector of a 16×16 block A of the input encoded data is5 rightwards and 4 upwards, the size of the block A is 32×32 after sizeconversion. In case the maximum unit size of the block for encoding is16×16, it is necessary for encoding that the block A sized 32×32following size conversion is divided into four parts. In this case, amotion vector of 10 rightwards and 8 upwards is sent to each of the four16×16 blocks resulting from the division. The same data values as thoseof the input encoded data, are used at this time for the reference frameinformation.

<Case 2> the Case of Halving the Image Size (×½)

In case the intra-frame prediction mode of a 4×4 block B of the inputencoded data is four leftwards, the block B after size conversion is2×2. In case the minimum unit size of the block for encoding at thistime is 4×4, it is necessary for encoding that the 2×2 block after sizeconversion is synthesized with neighboring 2×2 blocks to form a 4×4block. At this time, the prediction modes and the directions ofprediction of the 2×2 blocks are combined together to decide on a singleprediction mode.

Regarding combining method, such prediction mode and such direction ofprediction that are dominant in the four blocks are sent to the encodingunit 105. If, among the four 2×2 blocks, two are for leftwards and theremaining two are for upwards, with there being the mode of the 45°direction, a mode of 45° up leftward direction is selected. In case thefour 2×2 blocks are for leftward, rightward, upward and downward, anaverage value mode is used.

Except for those described above, the configuration and the operationare the same as those of the Exemplary Embodiments 1 to 3, and hence thedescription is dispensed with. The present Exemplary Embodiment has anadvantage that the image size of the input encoded data may be convertedexpeditiously, and that size conversion may be more flexible than inExemplary Embodiment 3.In the present Exemplary Embodiment, theprocessing by the conversion control unit 501, decoding unit 103, sizeconversion unit 104 and the encoding unit 105 may be implemented on aprogram which is executed on a computer of the moving image conversionapparatus 500.

Exemplary Embodiment 5

The following describes Exemplary Embodiment 5 of the present invention.FIG. 6 shows the configuration of a moving image conversion apparatus600 of Exemplary Embodiment 5 of the present invention. In FIG. 6, theelements that are the same as those of FIG. 1 are depicted by the samereference numerals. In the following, the explanation of the sameelements is fittingly dispensed with to avoid redundancy.

Based upon a size conversion request 107 from outside and encodingparameters 112 from a decoding unit 103, a conversion control unit 601sends a size conversion request 108 to a size conversion unit 104, andsends conversion information 603 to an encoding unit 602. As theconversion information 603, the following are sent.

-   -   an image data size;    -   frame type;    -   prediction mode and direction of prediction;    -   motion vector;    -   reference frame information;    -   block size;    -   block type; and    -   bit rate.

FIG. 7 shows a detailed configuration of an encoding unit 602 shown inFIG. 6. In FIG. 7, the elements that are the same as those of FIG. 2 aredepicted by the same reference numerals. In the following, theexplanation of the same elements is fittingly dispensed with to avoidredundancy.

Referring to FIG. 7, an inter-frame prediction encoding unit 701executes inter-frame prediction encoding using conversion information603 received from a conversion control unit 601. More specifically, theinter-frame prediction encoding unit 701 receives the reference frameinformation, a motion vector, a search range for the frame and a searchrange for the motion vector, as the conversion information 603.

On receipt of image data for one screen image from a frame memory 204,the inter-frame prediction encoding unit 701 executes inter-frameprediction for the image data of one screen image, just for the searchrange for the motion vector, with the motion vector as start point or asreference. The search range may be a range extending to a set of pointsevenly spaced apart a preset distance left wards, rightwards, upwardsand downwards from the motion vector as start point. Or, the inter-frameprediction encoding unit 701 executes search such as to take advantageof the motion vector direction. That is, the search is made to a pointspaced apart a preset distance from the above mentioned motion vector asstart point, just in the direction of the motion vector.

In case the frame search range is two or more frames, the inter-frameprediction encoding unit 701 executes motion vector search for a framesearch range, with a frame number as start point, in order to decide onbest motion vector and frame number 703. The motion vector and framenumber, thus decided on, are sent to a variable length encoder 210.

An intra-frame prediction encoding unit 702 executes intra-frameprediction/encoding, using the conversion information 603 received fromthe conversion control unit 601. More specifically, a prediction modeand a search range are received as the conversion information 603.

On receipt of a block for prediction and neighboring image data, theintra-frame prediction encoding unit 702 determines an optimum one 704,out of prediction modes in the search range, based on the receivedprediction mode as start point. The prediction mode 704, thus decidedon, is sent to the variable length encoder 210.

An example method for searching with the prediction mode received by theconversion information 603 as start point will now be described.

In case there are directions for prediction every 45°, and theprediction mode, received from the information 603, is for the leftwarddirection, with the search range being two, not only the intra-frameprediction in the leftward direction but the intra-frame prediction of45° up leftward as well as that of 45° down leftward is carried out todecide on an optimum prediction mode.

Except for those described above, the configuration and the operationare the same as those of the Exemplary Embodiments 1 to 4, and hence thecorresponding explanation is dispensed with. In the present ExemplaryEmbodiment, the image size of the input encoded data may speedily beconverted to advantage in such a manner as to enhance the image qualityin comparison with the Exemplary Embodiments 1 to 4. In the presentExemplary Embodiment, the processing carried out by the conversioncontrol unit 601, decoding unit 103, size conversion unit 104 and theencoding unit 602 may be implemented by a program which is executed on acomputer of the moving image conversion apparatus 600.

Exemplary Embodiment 6

The following describes Exemplary Embodiment 6 of the present invention.FIG. 8 shows a detailed configuration of a moving image conversionapparatus 800 in the Exemplary Embodiment 6. In FIG. 8, the elementswhich are the same as those of FIG. 1 are depicted by the same referencenumerals. In the following, the explanation of the same elements is notmade in order to avoid redundancy.

Based upon a size conversion request 107 from outside and upon encodingparameters 112 from a decoding unit 103, a conversion control unit 801sends a size conversion request 108 to a size conversion unit 104, andsends conversion information 803 to an encoding unit 802.

The conversion control unit 801 sends as the conversion information 803,information including:

-   -   image data size;    -   frame type;    -   prediction mode and direction of prediction;    -   motion vector;    -   reference frame information;    -   block size;    -   block type;    -   code amount distribution bit rate of input encoded data; and    -   threshold value to decide on whether or not to use a prediction        mode or a motion vector.

The threshold value may be exemplified by

-   -   (size of a block under prediction)×(mean value the amplitude of        image data or square value of the mean value of the amplitude of        image data), or    -   a target code amount in each block calculated from bit rate or        code amount distribution of the input encoded data.

FIG. 9 shows a detailed configuration of an encoding unit 802 inExemplary Embodiment 5 of the present invention shown in FIG. 8. In FIG.9, the elements which are the same as those of FIG. 2 are depicted bythe same reference numerals. In the following, the explanation of thesame elements is fittingly omitted to avoid redundancy.

Referring to FIG. 9, an inter-frame prediction data generation unit 901generates inter-frame prediction data, using a frame number and a motionvector in the conversion information 803 received from the conversioncontrol unit 801. The inter-frame prediction data generation unitcompares a difference between the inter-frame prediction data and theimage data in the frame memory with a threshold value. In case aprediction residual is less than the threshold value, the inter-frameprediction data generation unit 902 adopts the frame number and themotion vector received in the conversion information 803.

In case the prediction residual is greater than the threshold value, aninter-frame prediction encoding unit 902 executes inter-frame predictionto determine a motion vector and a frame number 906 for supply to avariable length encoder 210.

In determining a motion vector and a frame number, the motion vector andthe frame number, received in the conversion information 803, may beused as start point. An intra-frame prediction encoding unit 903 mayalso be used in place of the inter-frame prediction encoding unit 902.

An intra-frame prediction data generation unit 904 generates intra-frameprediction data, using the prediction mode received in the conversioninformation 803. The intra-frame prediction encoding unit 903 compares adifference between the intra-frame prediction data generated in theintra-frame prediction data generation unit 904 and image data in thevicinity of a block of interest with a threshold value.

In case the prediction error is less than the threshold value, theintra-frame prediction encoding unit 903 adopts the prediction modereceived in the conversion information 803.

In case the prediction residual is greater than the threshold value, theintra-frame prediction encoding unit 903 executes intra-frame predictionto determine a prediction mode 907, for supply to the variable lengthencoder 210. At this time, the prediction mode, received in theconversion information 803, may be used as start point. The inter-frameprediction encoding unit 902 may be used in place of the intra-frameprediction encoding unit 903.

Except for those described above, the configuration as well as theoperation is the same as in Exemplary Embodiments 1 to 5, and hence willnot be explained. In the present Exemplary Embodiment, the image size ofthe input encoded data may speedily be converted to advantage and, inaddition, the image quality may be enhanced in comparison with the caseof the Exemplary Embodiments 1 to 5. In the present ExemplaryEmbodiment, the processing by the conversion control unit 801, decodingunit 103, size conversion unit 104 and the encoding unit 802 may beimplemented by a program which is executed on a computer of the movingimage conversion apparatus 800.

Exemplary Embodiment 7

The following describes Exemplary Embodiment 7 of the present invention.FIG. 10 shows a detailed configuration of a moving image conversionapparatus 1000 in Exemplary Embodiment 7 of the present invention. InFIG. 10, the elements which are the same as those of FIG. 1 are depictedby the same reference numerals. In the following, the explanation of thesame elements is dispensed with in order to avoid redundancy.

Based upon a size conversion request 107 from outside and encodingparameters 112 output from a decoding unit 103, a conversion controlunit 1001 sends a size conversion request 108 to a size conversion unit104, and sends conversion information 1003 to an encoding unit 1002.

The conversion control unit 1001 sends to the to a size conversion unit104, as the conversion information 1003 information including:

-   -   image data size;    -   frame type;    -   prediction mode and direction of prediction;    -   motion vector;    -   reference frame information;    -   block size;    -   block type; and    -   quantization step size.

It is possible for the conversion control unit 1001 to compare at leastone of the image data size and the bit rate included in the sizeconversion request 107 supplied from outside, with the correspondinginformation before conversion, and to change a quantization step size toN or 1/N times that before conversion.

FIG. 11 shows a detailed configuration of an encoding unit 1002 in thepresent Exemplary Embodiment 7. In FIG. 11, the elements which are thesame as those of FIG. 9 are depicted by the same reference numerals. Inthe following, the explanation of the same elements is dispensed with inorder to avoid redundancy.

A quantizer 1101 executes quantization using the quantization step sizeincluded in the conversion information 1003 received from the conversioncontrol unit 1001.

Except for those described above, the configuration as well as theoperation is the same as in Exemplary Embodiments 1 to 6, and hence willnot be explained. In the present Exemplary Embodiment, the image size ofthe input encoded data may be speedily converted to advantage incomparison with the case of the Exemplary Embodiments 1 to 6. In thepresent Exemplary Embodiment, the processing by the conversion controlunit 1001, decoding unit 103, size conversion unit 104 and the encodingunit 102 may be implemented by a program which is executed on a computerof a moving image conversion apparatus 1000.

Exemplary Embodiment 8

The following describes Exemplary Embodiment 8 of the present invention.FIG. 12 shows a detailed configuration of a moving image conversionapparatus 1200 in Exemplary Embodiment 8 of the present invention. InFIG. 12, the elements which are the same as those of FIG. 1 are depictedby the same reference numerals. In the following, the explanation of thesame elements is fittingly dispensed with in order to avoid redundancy.

Based upon a size conversion request 107 from outside and encodingparameters 112 from a decoding unit 103, a conversion control unit 1201sends a size conversion request 108 to a size conversion unit 104, andsends conversion information 1203 to an encoding unit 1202.

The conversion control unit 1201 sends as the conversion information1203, for example:

-   -   image data size;    -   frame type;    -   prediction mode and direction of prediction;    -   motion vector;    -   reference frame information;    -   block size;    -   block type;    -   bit rate; and    -   distribution of the code amount of input encoded data.

FIG. 13 shows a detailed configuration of the encoding unit 1202 in thepresent Exemplary Embodiment 8. In FIG. 13, the elements which are thesame as those of FIG. 11 are depicted by the same reference numerals. Inthe following, the explanation of the same elements is dispensed with inorder to avoid redundancy.

Referring to FIG. 13, a quantizer 1301 determines a target code amountof each block, using the distribution of the code amount of the inputencoded data and the bit rate included in the conversion information1203, received from the conversion control unit 1201. The quantizer 1301accordingly determines the quantization step size to execute thequantization.

Except for those described above, the configuration as well as theoperation is the same as in Exemplary Embodiments 1 to 7, and hence willnot be explained. In the present Exemplary Embodiment, the image size ofthe input encoded data may be adjusted to advantage more satisfactorilyin comparison with the case of the Exemplary Embodiments 1 to 7. In thepresent Exemplary Embodiment, the processing by the conversion controlunit 1201, decoding unit 103, size conversion unit 104 and the encodingunit 1202 may be implemented by a program which is executed on acomputer of the moving image conversion apparatus 1200.

Exemplary Embodiment 9

The following describes Exemplary Embodiment 9 of the present invention.FIG. 14 shows a detailed configuration of a moving image conversionapparatus 1400 in Exemplary Embodiment 9 of the present invention. InFIG. 14, the elements which are the same as those of FIG. 1 are depictedby the same reference numerals. In the following, the explanation of thesame elements is fittingly dispensed with in order to avoid redundancy.

Based upon a size conversion request 107 from outside and upon encodingparameters 112 from a decoding unit 103, a conversion control unit 1401sends a size conversion request 108 to a size conversion unit 104, andsends conversion information 1403 to an encoding unit 1402.

As the conversion information 1403, image data size, frame type,prediction mode with direction of prediction, motion vector, referenceframe information, block size, block type, bit rate and quantizationinformation, are sent.

The quantization information includes information on whether or not touse the quantization step size of input encoded data or the distributionof the code amount of the input encoded data, information onquantization step size and information on bit rate.

In giving a decision on whether or not to use the quantization step sizeof the input encoded data, the quantization step size of input encodeddata is not used in cases of a greater size conversion ratio, a greaterratio between the bit rate of the input encoded data and that of outputencoded data or a greater difference between the increase/decrease ofthe size conversion ratio and the increase/decrease of the input/outputbit rate ratio.

In giving a decision on whether or not to use the code amountdistribution of the input encoded data, such code amount distribution isnot used in case there is a greater difference present between thetarget bit rate and the output code amount.

FIG. 15 shows a detailed configuration of the encoding unit 1402 inExemplary Embodiment 9 of the present invention. In FIG. 15, theelements which are the same as those of FIG. 9 are depicted by the samereference numerals. In the following, the explanation of the sameelements is fittingly dispensed with in order to avoid redundancy.

A quantizer 1501 executes quantization using information included in theconversion information 1403 received from the conversion control unit1401. The conversion information 1403 may include the information onwhether or not to use the quantization step size of input encoded dataor code amount distribution, on quantization step size, on code amountdistribution and on bit rate.

Specifically, the quantizer executes quantization based upon theinformation on whether or not to use the code amount distribution or thequantization step size of the input encoded data. If the code amountdistribution or the quantization step size is to be used, the quantizer1501 executes the quantization using the quantization step size of theinput encoded data or the quantization step size obtained bycalculations using the code amount distribution. If conversely the codeamount distribution or the quantization step size is not used, thequantizer sets the quantization step size on its own based on the bitrate and accordingly executes the quantization.

Except for those described above, the configuration as well as theoperation is the same as in Exemplary Embodiments 1 to 8, and hence isnot explained. In the present Exemplary Embodiment, the image size ofthe input encoded data may speedily be converted to advantage and, inaddition, the image quality may be enhanced in comparison with the caseof the Exemplary Embodiment 7 or 8. In the present Exemplary Embodiment,the processing by the conversion control unit 1401, decoding unit 103,size conversion unit 104 and the encoder 1402 may be implemented by aprogram which is executed on a computer of the moving image conversionapparatus 1400.

Exemplary Embodiment 10

The following describes Exemplary Embodiment 10 of the presentinvention. FIG. 16 shows a detailed configuration of a moving imageconversion apparatus 1600 of Exemplary Embodiment 10 of the presentinvention.

Based upon a size conversion request 107 from outside, and encodingparameters 112 from a decoding unit 103, a conversion control unit 1601sends a size conversion request 108 to a size conversion unit 104, andsends conversion information 1603 to an encoding unit 1602.

As the conversion information 1603, image data size, frame type,prediction mode with direction of prediction, motion vector, referenceframe information, block size, block type, bit rate and filterprocessing information are sent.

The filter processing information may be exemplified by information onwhether or not to execute filtering processing on image data beforeencoding or on image data being encoded and, in case of executingfiltering processing, the information on the intensity of filtering oron an area where filtering is to be executed.

A decision on whether or not the filtering processing is to be executed,resort may be made using information on output bit rate with respect toimage size after size conversion, information on whether the image sizeis to be enlarged or contracted, or information on whether bit rate ofthe input encoded data is high or low.

For example, a decision for executing or not executing the filteringprocessing, resort may be made using information in the input encodeddata indicating whether the filtering processing is or is not to beexecuted. A decision for executing the filtering processing may be madein case of a low bit rate of the input encoded data or a low output bitrate with respect to the image size after size conversion. A decisionfor not executing the filtering processing may be made in case of a highbit rate of the input encoded data or a high output bit rate withrespect to the image size after size conversion, or in case of reducingthe image size.

FIG. 17 shows a detailed configuration of the encoding unit 1602 inExemplary Embodiment 10 of the present invention. In FIG. 17, theelements which are the same as those of FIG. 9 are depicted by the samereference numerals. In the following, the explanation of the sameelements is dispensed with in order to avoid redundancy.

An in-loop filter 1701 executes filtering based on the conversioninformation 1603 received from the conversion control unit 1601. Wheninstructed by the conversion information 1603 to execute the filtering,the in-loop filter 1701 executes filtering processing on image data1702.

If there is information in the conversion information 1603 on filteringintensity or on an area where filtering is to be performed, the in-loopfilter 1701 executes the filtering processing in accordance with theconversion information 1603.

If instructed by the conversion information 1603 not to execute thefiltering, the in-loop filter 1701 sends the image data 1702 as it is toa frame memory 204.

In the present Exemplary Embodiment, executing or not executing thefiltering processing on image data being encoded has been explained. Itis to be understood that this explanation is only by way of illustrationof the present invention and not for limiting the invention. As will beapparent from the principle of the moving image conversion apparatus ofthe present invention, the foregoing explanation may apply in giving adecision on whether or not the filtering processing is to be executed onpre-encoding image data as well.

Except for those described above, the configuration as well as theoperation is the same as in Exemplary Embodiments 1 to 9, and hence isnot explained. In the present Exemplary Embodiment, the image size ofthe input encoded data may be converted to advantage expeditiously and,in addition, the image quality may be enhanced in comparison with thecase of the Exemplary Embodiments 1 to 9. In the present ExemplaryEmbodiment, the processing by the conversion control unit 1601, decodingunit 103, size conversion unit 104 and the encoder 1602 may beimplemented by a program which is executed on a computer of the movingimage conversion apparatus 1600.

Exemplary Embodiment 11

The following describes Exemplary Embodiment 11 of the presentinvention. FIG. 18 shows a detailed configuration of a moving imageconversion apparatus 1800 in Exemplary Embodiment 11 of the presentinvention. Based upon a size conversion request 1803 from outside andencoding parameters 112 from a decoding unit 103, a conversion controlunit 1801 sends conversion information 1804 to a size conversion unit104, and sends conversion information 1805 to an encoding unit 1802.

The conversion control unit 1801 sends the conversion information 1804,inclusive of a request for size conversion and for frame skip, to thesize conversion unit 104, based upon the size conversion request 1803.

The conversion information 1804 includes;

-   -   multiplication factor;    -   image size after size conversion;    -   bit rate;    -   encoding information; and    -   frame rate information.

The frame rate information includes:

-   -   frame rate information of both of input encoded data and output        encoded data;    -   the number of frames to be decimated of the input encoded data;        and    -   timing information.

On receipt of the encoding parameters 112 from the decoding unit 103,the conversion control unit 1801 sends the conversion information 1805to the encoding unit 1802, based upon the size conversion request 1803.

The conversion information 1805 includes:

-   -   image data size;    -   frame type;    -   prediction mode and direction of prediction;    -   motion vector;    -   reference frame information;    -   block size;    -   block type;    -   bit rate;    -   quantization information;    -   filtering processing information; and    -   frame skip information.

The frame skip information includes:

-   -   frame rate information for both of input encoded data and output        encoded data;    -   information on the number of frames of the input encoded data to        be decimated; and    -   timing information.

The conversion control unit 1801 determines whether or not the framerate is to be dropped, and notifies the encoding unit 1802 of whether ornot to execute encoding on a per frame basis.

Taking an example case of doubling the image size and decimating everyother frame, by way of dropping the frame rate, the method of generatinga motion vector and the reference frame information will now beexplained. It should be noted however that, as will be apparent from theprinciple of the present invention, the size as well as the frame ratedifferent from those given above may also be used.

<Case 1>

Regarding the reference frame, if a frame in question references a frameto be decimated, a past output fame or a frame to be output in futurewith respect to the frame being decimated is referenced. Since it isalso necessary to decimate the reference frame number, the frame numberis decremented a number equal to the number of frames to be decimated.In case the frame in question is referencing a frame to be output, theframe to be output is referenced in an unchanged state. Since it isnecessary to decimate the frame number in this case as well, the framenumber is decremented a number equal to the number of the frames to bedecimated.

Regarding the motion vector, since the image size is doubled, the motionvector of the input encoded data is doubled. If the frame in question isreferencing a frame to be output, the motion vector size is to remainequal to a value indicated by the image size ratio.

If a frame in question references a frame to be decimated, a past outputfame or a frame to be output in future with respect to the frame to bedecimated is referenced. For example, if the frame to be decimated is aframe one frame before the frame in question, and a frame two framesbefore the frame in question is referenced, the motion vector is madeequal to two times a value as indicated from the image size ratio.

For example, if the frame to be decimated is a frame three frames beforethe frame in question, and a frame which is two frames before the framein question is referenced, the motion vector is further made equal to0.66 times a value as indicated from the image size ratio.

<Case 2>

Regarding a reference frame, if the frame in question is referencing aframe to be decimated, a frame to be output, referenced by the frame tobe decimated, is referenced. Since it is necessary to decimate thereference frame number as well, the frame number is decremented a numberequal to the number of the frames to be decimated.

In case the frame in question is referencing a frame to be output, theframe to be output is referenced in an unchanged state. Since it isnecessary to decimate the reference frame number in this case as well,the frame number is decremented a number equal to the number of theframes to be decimated.

Turning now to the motion vector, since the image size is doubled, themotion vector of the input encoded data is doubled. In case the frame inquestion is referencing a frame to be output, the size of the motionvector is to remain equal to a value indicated by the image size ratio.

In case the frame in question is referencing a frame to be decimated, apast output fame or a frame to be output in future with respect to theframe being decimated is referenced. The motion vector in the inputencoded data of the frame in question is summed to the motion vector ofthe frame to be decimated.

If notified by the conversion control unit 1801 of the conversioninformation 1804 including a frame skip request, the size conversionunit 104 discards image data 113, supplied thereto from the decodingunit 103, without converting its size. That is, the size conversion unitdoes not transmit the image data to the encoding unit 1802.

The encoding unit 1802 generates output encoded data based upon theconversion information 1805 and image data 114. Except for thosedescribed above, the configuration as well as the operation is the sameas in the Exemplary Embodiments 1 to 10, and hence will not beexplained.

With the present Exemplary Embodiment, it is possible to speedily changethe image size of the input encoded data in case of changing the framerates of the input encoded data and the output encoded data. In thepresent Exemplary Embodiment, the processing in the conversion controlunit 1801, decoding unit 103, size conversion unit 104 and the encodingunit 1802 may be implemented by a program which is executed on acomputer of the moving image conversion apparatus 1800.

Exemplary Embodiment 12

The following describes Exemplary Embodiment 12 of the presentinvention. FIG. 19 shows a detailed configuration of a moving imageconversion apparatus 1900 in Exemplary Embodiment 12 of the presentinvention. In FIG. 19, the elements which are the same as those of FIG.1 are depicted by the same reference numerals. In the following, theexplanation of the same elements is dispensed with in order to avoidredundancy.

Based upon a size conversion request 1906 from outside and encodingparameters 112 from a decoding unit 1902, a conversion control unit 1901sends a size conversion request 1904 to a size conversion unit 104, andsends conversion information 1905 to an encoding unit 1903.

The size conversion request 1904 includes size conversion information aswell as information regarding the encoding scheme for output encodeddata and that regarding the encoding scheme for input encoded data.

On receipt of encoding parameters 112 from the decoding unit 1902, theconversion control unit 1901 sends image data size, frame type,prediction mode with direction of prediction, motion vector, referenceframe information, block size, block type, bit rate, quantizationinformation and filtering processing information, to the encoding unit1903, based upon the size conversion request 1904.

If, in the size conversion request 1904, the input encoding scheme isthe same as the output encoding scheme, the conversion control unit 1901converts the encoding parameter 112 by a method explained in the abovementioned Exemplary Embodiments 1 to 11. The conversion control unitthen supplies the so converted encoding parameters 112 to the encodingunit 1903.

If, in the size conversion request 1904, the input encoding scheme isdifferent from the output encoding scheme, the conversion control unit1901 converts the prediction mode with direction of prediction, motionvector, reference frame information, block size, block type, picturetype, quantization information and filtering processing information sothat they fit in with those of the output encoding scheme.

As regards the prediction mode, if the prediction mode for the inputencoding scheme is not matched to that of the output encoding scheme,the prediction mode and the direction of prediction are found based onapproximation to the closet prediction mode and direction.

As regards the motion vector, its upper/lower limit values and measureare adjusted for matching to those of the output encoding scheme

As regards the reference frame information, if say the reference frameinformation of the input encoding scheme is not matched in the outputencoding scheme, the frame number is increased or decreased to themaximum value or the minimum value of the output encoding scheme.

The reference frame information irrelevant to the output encoding schemeis made not to be sent to the encoding unit 1903. Or, relevantinformation portions are re-encoded without using the information of theinput encoded data.

As regards the block size, there may be a case where a single predictionmode/prediction direction as well as a single motion vector may bespecified for each of 4×4 blocks in the input encoding scheme but just asingle prediction mode/prediction direction or just a single motionvector may be specified for an 8×8 block or for a 16×16 block in theoutput encoding scheme. In such case, the prediction mode/predictiondirection or the motion vector is synthesized as shown in the ExemplaryEmbodiment 4.

As regards the picture type, if a certain picture type is present in theinput encoding scheme but is not present in the output encoding scheme,the frame in question is re-encoded. Or, if the picture type differenceis merely the addition of the information irrelevant to the outputencoding scheme, the information is not sent but the picture type isapproximated and sent to the encoding unit 1903.

As regards the quantization step, it is converted as the upper/lowerlimit values or the meaning of the measure, for example, whether thescale is a log scale or whether the number is a multiple number, istaken into account.

As regards the filtering processing information, no filtering operationis carried out if the information is present in the input encodingscheme and is not present in the output encoding scheme. Or, if simplythe information irrelevant to the output encoding scheme is added, suchinformation is not sent and the filtering processing information isgenerated.

Except for those described above, the configuration and the operationare the same as those of the above Exemplary Embodiments 1 to 11, andhence the explanation is dispensed with. In the present ExemplaryEmbodiment, it is possible to speedily change the image size of theinput encoded data in case the input encoding scheme differs from theoutput encoding scheme. In the present Exemplary Embodiment, theprocessing by the conversion control unit 1901, decoding unit 1902, sizeconversion unit 104 or the encoding unit 1903 may be implemented by aprogram which is executed on a computer of the moving image conversionapparatus 1900.

The whole or part of the Exemplary Embodiments disclosed above can bedescribed as, but not limited to, the following Supplementary notes.

(Supplementary Note 1)

A image conversion apparatus comprising:

-   -   means that decodes encoded data received;    -   means that changes the size of image data decoded; and    -   means that re-uses, in encoding the image data after size        change, at least one of a prediction mode, a block type, a        motion vector and reference frame information of the encoded        data received, for the encoding.

(Supplementary Note 2)

The image conversion apparatus according to Supplementary note 1,further comprising:

-   -   means that expands or contracts, in encoding the image data        after size change, at least one of the motion vector and the        block type of the encoded data received, according to image size        ratio between the image size before the size change and that        after the size change.

(Supplementary Note 3)

The image conversion apparatus according to Supplementary note 1,further comprising:

-   -   means that approximates, in encoding the image data after the        size change, the prediction mode, in case the prediction mode of        the encoded data received is unable to be used as it is after        the size change.

(Supplementary Note 4)

The image conversion apparatus according to any one of Supplementarynotes 1 to 3, further comprising:

-   -   means that splits or synthesizes at least one of the prediction        mode and the motion vector, for re-using, according to image        size ratio between the image size before the size change and        that after the size change.

(Supplementary Note 5)

The image conversion apparatus according to any one of Supplementarynotes 1 to 4, further comprising:

-   -   means that executes re-prediction, using, as start point, at        least one of the motion vector and the prediction mode re-used.

(Supplementary Note 6)

The image conversion apparatus according to Supplementary note 5,further comprising:

-   -   means that uses, in executing re-prediction with the re-used        motion vector as start point, direction of the motion vector.

(Supplementary Note 7)

The image conversion, apparatus according to any one of Supplementarynotes 1 to 6, further comprising:

-   -   means that executes the re-prediction in case prediction error        is not less than a predetermined threshold value.

(Supplementary Note 8)

The image conversion apparatus according to any one of Supplementarynotes 1 to 7, further comprising:

-   -   means that re-uses, in encoding the image data after size        change, quantization step size of received encoded data.

(Supplementary Note 9)

The image conversion apparatus according to any one of Supplementarynotes 1 to 7, further comprising:

-   -   means that re-uses, in encoding the image data after size        change, code amount distribution of the received encoded data.

(Supplementary Note 10)

The image conversion apparatus according to Supplementary note 8 or 9,further comprising:

-   -   means that determines, in encoding the image data after size        change, whether or not at least one of quantization step size        and code amount distribution of the input image data is to be        re-used.

(Supplementary Note 11)

The image conversion apparatus according to any one of Supplementarynotes 1 to 10, further comprising:

-   -   means that changes, in encoding the image data after size        change, bit rate of the received encoded data.

(Supplementary Note 12)

The image conversion apparatus according to any one of Supplementarynotes 1 to 11, further comprising:

-   -   means that determines, in encoding the image data after size        change, whether or not filtering is to be applied to at least        one of the image data after the size change and the image data        being encoded.

(Supplementary Note 13)

The image conversion apparatus according to any one of Supplementarynotes 1 to 12, further comprising:

-   -   means that re-uses, in encoding the image data after size        change, picture type of the received encoded data.

(Supplementary Note 14)

The image conversion apparatus according to any one of Supplementarynotes 1 to 13, wherein frame rate is converted in encoding the imagedata after size change.

(Supplementary Note 15)

The image conversion apparatus according to any one of Supplementarynotes 1 to 14, wherein decoding scheme and encoding scheme are differentfrom each other.

(Supplementary Note 16)

A image conversion method comprising:

-   -   decoding encoded data received into image data;    -   changing size of the image data decoded; and    -   re-using, in encoding the image data after size conversion, at        least one of prediction mode, block type, motion vector and        reference frame information of the encoded data received, for        the encoding.

(Supplementary Note 17)

The image conversion method according to Supplementary note 16, furthercomprising:

-   -   expanding or contracting, in encoding the image data after size        change, at least one of the motion vector and the block type of        the encoded data received, for matching to a size ratio between        the image size before the size change and that after the size        conversion.

(Supplementary Note 18)

The image conversion method according to Supplementary note 16, furthercomprising:

-   -   approximating, in encoding the image data after the size change,        the prediction mode in case the prediction mode of the encoded        data received is unable to be used as it is after the size        conversion.

(Supplementary Note 19)

The image conversion method according to any one of Supplementary notes16 to 18, further comprising:

-   -   splitting or synthesizing at least one of the prediction mode        and the motion vector, for re-using, for matching to a size        ratio between the image size before the size conversion and that        after the size change.

(Supplementary Note 20)

The image conversion method according to any one of Supplementary notes16 to 19, further comprising:

-   -   determining, in encoding the image data after size change,        whether or not at least one of quantization step size and code        amount distribution of the input encoded data is to be re-used.

(Supplementary Note 21)

The image conversion method according to any one of Supplementary notes16 to 20, further comprising

-   -   converting frame rate in encoding the image data after size        change.

(Supplementary Note 22)

A program causing a computer to execute the processing comprising:

-   -   decoding encoded data received;    -   changing size of image data decoded; and    -   re-using, in encoding the image data having size thereof        changed, at least one of prediction mode, block type, motion        vector and reference frame information of the encoded data        received, for the encoding.

(Supplementary Note 23)

The program according to the Supplementary note 22 causing the computerto execute the processing comprising

-   -   expanding or contracting, in encoding the image data after size        change, at least one of the motion vector and the block type of        the encoded data received, for matching to a size ratio between        the image size before the size change and that after the size        change.

(Supplementary Note 24)

The program according to the Supplementary note 22 causing the computerto execute the processing comprising

-   -   approximating, in encoding the image data after the size change,        the prediction mode in case the prediction mode of the encoded        data received is unable to be directly used after the size        change.

(Supplementary Note 25)

The program according to any one of the Supplementary notes 22 to 24causing the computer to execute the processing comprising

-   -   splitting or synthesizing at least one of the prediction mode        and the motion vector, for re-using, according to size ratio        between the image size before the size change and that after the        size change.

(Supplementary Note 26)

The program according to any one of the Supplementary notes 22 to 25causing the computer to execute the processing comprising

-   -   determining, in encoding the image data after size change,        whether or not at least one of the quantization step size and        the distribution of the code amount of the input image data is        to be re-used.

(Supplementary Note 27)

The program according to any one of the Supplementary notes 22 to 26causing the computer to execute the processing comprising

-   -   converting the frame rate in encoding the image data after size        change.

(Supplementary Note 28)

A image conversion apparatus comprising:

-   -   a decoding unit that decodes encoded data received into image        data;    -   a conversion control unit that receives a size conversion        request from outside the apparatus or an encoding parameter set        from the decoding unit to control a size conversion unit or an        encoding unit;    -   a size conversion unit that executes size conversion of the        image data entered from the decoding unit under a command from        the conversion control unit; and    -   an encoding unit that generates encoded data from the image data        entered from the size conversion unit, under a command from the        conversion control unit;    -   the encoding unit, in encoding the image data having size        thereof changed by the size conversion unit, re-using, at least        one of prediction mode, block type, motion vector and reference        frame information of the encoded data received.

(Supplementary Note 29)

The image conversion apparatus according to claim 28, wherein theconversion control unit sends a size conversion request to the sizeconversion unit and sends conversion information to the encoding unit,based upon the size conversion request entered from outside theapparatus, and upon the encoding parameter from the decoding unit;

-   -   the size conversion unit converting the image size of image data        received from the decoding unit, using a multiplication factor        or an input/output image size in size conversion request        information from the conversion control unit, to deliver the        image data after size conversion to the encoding unit;    -   the encoding unit including    -   an inter-frame prediction data generation unit that executes        inter-frame prediction, using the conversion information        received from the conversion control unit and image data from a        frame memory, to generate prediction data which is supplied to a        switch;    -   an intra-frame prediction data generation unit that executes        intra-frame prediction, using the conversion information        received from the conversion control unit and the image data        received from the size conversion unit, to generate prediction        data which is supplied to a switch;    -   the switch changing over between the prediction data generated        by the inter-frame prediction data generation unit and the        prediction data generated by the intra-frame prediction data        generation unit, in accordance with a command from the        conversion control unit, so that the prediction data will be        able to be sent to a discrete cosine transform unit, an in-loop        filter and to the intra-frame prediction data generation unit;    -   a quantizer that determines the quantization step with a bit        rate commanded by the conversion control unit as a goal;    -   a header information generator that generates the header        information based upon the conversion information received from        the conversion control unit; and    -   a variable length encoder that variable length encodes an output        of the quantizer based upon the header information from the        header information generator.

The disclosures of the Patent Documents 1 to 4 are to be incorporatedherein by reference. The particular exemplary embodiments or examplesmay be modified or adjusted within the gamut of the entire disclosure ofthe present invention, inclusive of claims, based on the fundamentaltechnical concept of the invention. Further, a wide variety ofcombinations or selection of elements disclosed herein may be madewithin the scope of coverage by the claims. That is, the presentinvention may encompass various modifications or corrections that mayoccur to those skilled in the art in accordance with and within theentire disclosure of the present invention, inclusive of claims, and thetechnical concept of the present invention.

EXPLANATION OF REFERENCE NUMERALS

100 moving image conversion apparatus

101 conversion control unit

102 reception buffer

103 decoding unit

104 size conversion unit

105 encoding unit

106 transmission buffer

107 size conversion request

108 size conversion request

109 conversion information

110 input encoded data

111 input encoded data

112 set of encoding parameters

113 image data

114 image data

115 encoded data

116 encoded data

117 encoding information

201 inter-frame prediction data generation unit

202 intra-frame prediction data generation unit

203 switch

204 frame memory

205 DCT (Discrete Cosine Transform) unit

206 quantizer

207 inverse quantizer

208 inverse DCT unit

209 in-loop filter

210 variable length encoder

211 header information generator

212 header information

213 image data

300 moving image conversion apparatus

301 conversion control unit

400 moving image conversion apparatus

401 conversion control unit

500 moving image conversion apparatus

501 conversion control unit

600 moving image conversion apparatus

601 conversion control unit

602 encoding unit

603 conversion information

701 inter-frame prediction encoding unit

702 intra-frame prediction encoding unit

703 motion vector and reference frame information

704 prediction mode

800 moving image conversion apparatus

801 conversion control unit

802 encoding unit

803 conversion information

901 inter-frame prediction data generation unit

902 inter-frame prediction encoding unit

903 intra-frame prediction data generation unit

904 intra-frame prediction encoding unit

905 switch

906 motion vector and reference frame information

907 prediction mode

1000 moving image conversion apparatus

1001 conversion control unit

1002 encoding unit

1003 conversion information

1101 quantizer

1200 moving image conversion apparatus

1201 conversion control unit

1202 encoding unit

1203 conversion information

1301 quantizer

1400 moving image conversion apparatus

1401 conversion control unit

1402 encoding unit

1403 conversion information

1501 quantizer

1600 moving image conversion apparatus

1601 conversion control unit

1602 encoding unit

1603 conversion information

1701 in-loop filter

1702 image data

1800 moving image conversion apparatus

1801 conversion control unit

1802 conversion request

1803 size conversion/frame skip request

1804 conversion information

1900 moving image conversion apparatus

1901 conversion control unit

1902 decoding unit

1903 encoding unit

1904 conversion request

1905 conversion information

1. An image conversion apparatus comprising: a decoding unit thatdecodes encoded data received; a size conversion unit that changes sizeof image data decoded by said decoding unit; an encoding unit that inencoding said image data having size thereof changed by said sizeconversion unit, re-uses for encoding, at least one of prediction mode,block type, motion vector and reference frame information of saidencoded data received; and a conversion control unit that receives asize conversion request from outside said apparatus and a set ofencoding parameters supplied from said decoding unit to control saidsize conversion unit and said encoding unit.
 2. The image conversionapparatus according to claim 1, wherein in encoding said image datahaving size thereof changed, said size conversion unit expands orcontracts at least one of said motion vector and said block type of saidencoded data received, according to image size ratio between said imagesize before size change and that after size change, based on a sizeconversion request supplied from said conversion control unit.
 3. Theimage conversion apparatus according to claim 1, wherein in encodingsaid image data having size thereof changed, said conversion controlunit approximates said prediction mode, in case said prediction mode ofsaid encoded data received supplied as said encoding parameter from saiddecoding unit is unable to be used just as it is after said size change.4. The image conversion apparatus according to claim 1, wherein saidsize conversion unit splits or synthesizes, at least one of saidprediction mode and said motion vector for re-using, according to imagesize ratio between said image size before said size change and thatafter said size change, based on a size conversion request supplied fromsaid conversion control unit.
 5. The image conversion apparatusaccording to claim 1, wherein said encoding unit executes re-prediction,using at least one of said re-used prediction mode and said motionvector as start point, based on conversion information supplied theretoby said conversion control unit.
 6. The image conversion apparatusaccording to claim 5, wherein said encoding unit uses said direction ofsaid motion vector in executing said re-prediction with said re-usedmotion vector as start point, based on conversion information suppliedthereto by said conversion control unit.
 7. The image conversionapparatus according to claim 1, wherein said encoding unit executesre-prediction in case said prediction error is not less than apredetermined threshold value, based on conversion information suppliedthereto by said conversion control unit.
 8. The image conversionapparatus according to claim 1, wherein said encoding unit a quantizerthat re-uses, in encoding said image data having size thereof changed,quantization step size of said encoded data received, based onconversion information supplied thereto by said conversion control unit.9. The image conversion apparatus according to claim 1, wherein saidencoding unit a quantizer that re-uses, in encoding said image datahaving size thereof changed, code amount distribution of said encodeddata received, based on conversion information supplied thereto by saidconversion control unit.
 10. The image conversion apparatus according toclaim 8, wherein said conversion control unit determines whether or notat least one of said quantization step size and said code amountdistribution of said input image data is to be re-used in encoding saidimage data having size thereof changed.
 11. The image conversionapparatus according to claim 1, wherein said encoding unit changes, inencoding said image data having size thereof changed by said sizeconversion unit, bit rate of said encoded data received, based onconversion information supplied thereto by said conversion control unit.12. The image conversion apparatus according to claim 1, wherein saidconversion control unit determines whether or not filtering is to beapplied to at least one of said image data having size thereof changedby said size conversion unit and image data which is being encoded bysaid encoding unit.
 13. The image conversion apparatus according toclaim 1, wherein said encoding unit re-uses, in encoding said image datahaving size thereof changed, picture type of said encoded data received,based on conversion information supplied thereto by said conversioncontrol unit.
 14. The image conversion apparatus according to claim 1,wherein said encoding unit, in encoding said image data having sizethereof changed, converts frame rate.
 15. The image conversion apparatusaccording to claim 1, wherein decoding scheme adopted in said decodingunit and encoding scheme adopted in said encoding unit are differentfrom each other.
 16. An image conversion method comprising: decodingencoded data received into image data; performing size conversion ofsaid image data decoded; re-using for encoding, at least one ofprediction mode, block type, motion vector, and reference frameinformation of said encoded data received, in encoding said image datahaving size thereof converted; and controlling said size conversion andsaid encoding, based on size conversion request received from outsideand encoding parameter from said decoding processing.
 17. A computerreadable-recording medium storing therein a program causing a computerto execute the processing comprising: decoding encoded data received;performing size conversion of image data decoded; re-using for encoding,at least one of prediction mode, block type, motion vector and referenceframe information of said encoded data received, in encoding said imagedata having size thereof changed by said size conversion unit; andcontrolling said size conversion and said encoding, based on sizeconversion request received from outside and encoding parameter fromsaid decoding processing.